METHOD AND APPARATUS FOR COMPOSING AND PERFORMING 

MUSIC 

CROSS-REFERENCE TO RELATED APPLICATION(S) 

[001] This application claims priority to U.S. Provisional Patent 

Application No. 60/391,838, filed June 26, 2002, which is incorporated herein by 
reference in its entirety. 

TECHNICAL FIELD 

[002] The present invention relates generally to the field of musical 

apparatus. More specifically, the present invention relates to a musical 
performance and composition apparatus incorporating a user interface that is 
adaptable for use by individuals with physical disabilities. 

BACKGROUND OF THE INVENTION 

[003] Teaching music performance and composition to individuals with 

physical and mental disabilities requires special adaptive equipment. Currently, 
these individuals have limited opportunities to learn to perform and compose their 
own music because of the unavailability of musical equipment that is adaptable 
for their use. Teaching music composition and performance to individuals with 
physical and mental disabilities requires instruments and teaching tools that are 
designed to compensate for disabled students' limited physical and cognitive 
abilities. 

[004] For example, students with physical and mental disabilities such as 

cerebral palsy often have extremely limited manual dexterity and thus are unable 
to play the typical keyboard instrument with a relatively large number of narrow 
keys. Similarly, a user with physical disabilities may have great difficulty 
grasping and manipulating drumsticks and thus would be unable to play the 
typical percussion device. Also, disabled users are unable to accurately control 
the movements of their hands, which, combined with an extremely limited range 



of motion, can also substantially limit their ability to play keyboard, percussion, 
or other instruments. Such users may, however, exhibit greater motor control 
using their head or legs. 

[005] Furthermore, the currently available musical instruments are 

generally inflexible in regard to the configurations of their user interfaces. For 
example, keyboards typically have a fixed number that cannot be modified to 
adapt to the varying physical capabilities of different users. In addition, 
individuals with cognitive delays are easily distracted and can lose focus when 
presented with an overwhelming number of keys. Similarly, teaching individuals 
with mental and physical disabilities basic music theory requires a music tutorial 
device that has sufficient flexibility to adjust for a range of different cognitive 
abilities. 

[006] Consequently, there is a need in the art for a music performance 

and composition apparatus with a user interface adaptable for use by individuals 
with physical and mental disabilities, such that these individuals can perform and 
compose music with minimal involvement by others. In addition, there is a need 
for an apparatus allowing disabled users to use the greater motor control available 
in their head or legs. Furthermore, there is a need in the art for a music 
composition and performance tutorial system incorporating this new apparatus 
that allows musicians with disabilities to learn to compose and perform their own 
music. 

BRIEF SUMMARY OF THE INVENTION 

[007] The present invention, in one embodiment, is an interactive music 

apparatus. The apparatus has at least one actuator, a voltage converter, a 
processing computer, a speaker, and an output component. The actuator is 
configured to transmit a signal upon actuation and the voltage converter is 
configured to convert the signal from the actuator into a data stream. The 
processing computer is configured to convert the data stream into a first output 
signal and a second output signal. The speaker is configured to receive the first 



output signal and emit sound. The output component is configured to receive the 
second output signal and perform an action based on the second output signal. 
[008] According to a further embodiment, the present invention is a 

method of music performance and composition. The method includes actuating 
transmission of a signal, converting the signal into a data stream, converting the 
data stream at a processing computer into a first output signal and a second output 
signal, emitting sound at a speaker based on the first output signal, and 
performing an action at an output component based on the second output signal. 
[009] While multiple embodiments are disclosed, still other 

embodiments of the present invention will become apparent to those skilled in the 
art from the following detailed description, which shows and describes illustrative 
embodiments of the invention. As will be realized, the invention is capable of 
modifications in various obvious aspects, all without departing from the spirit and 
scope of the present invention. Accordingly, the drawings and detailed 
description are to be regarded as illustrative in nature and not restrictive. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[010] FIG. 1 is a schematic diagram of one embodiment of the present 

invention. 

[011] FIG. 1A is a schematic diagram of an alternative embodiment of 

the present invention. 

[012] FIG. 2 is a flow chart showing the operation of the apparatus, 

according to one embodiment of the present invention. 

[013] FIG. 2A is a flow chart depicting the process of launching a web 

browser using the apparatus, according to one embodiment of the present 
invention. 

[014] FIG. 2B is a flow chart depicting the process of displaying a 

graphical keyboard using the apparatus, according to one embodiment of the 
present invention. 



[015] FIG. 2C is a flow chart depicting the process of displaying a music 

staff using the apparatus, according to one embodiment of the present invention. 
[016] FIG. 2D is a flow chart depicting the process of providing a 

display of light using the apparatus, according to one embodiment of the present 
5 invention. 

[017] FIG. 3 is a schematic diagram of a voltage controller, according to 

one embodiment of the present invention. 

[018] FIG. 4 is a perspective view of a user console and an optional 

support means, according to one embodiment of the present invention. 
10 [019] FIG. 5 is a cross-section view of a user interface board according 

to one embodiment of the present invention. 

DETAILED DESCRIPTION 

[020] FIG. 1 shows a schematic diagram a music apparatus 10, according 

to one embodiment of the present invention. As shown in FIG. 1, the music 

15 apparatus 10 may include a user console 20 having at least one actuator 30 with 

an actuator button 31, a voltage converter 100, a processing computer 150 having 
a processor 154 software 152, and an internal sound card 148, a display monitor 
180, and a speaker 159. In a further embodiment, the voltage converter 100 is an 
integral component of the user console 20. The actuator 30 is connected to the 

20 voltage converter 100 with an actuator cable 35. The voltage converter is 

connected to the processing computer 150 with a serial cable 145. The processing 
computer 150 is connected to the display monitor 180 by a monitor cable 177. 
The processing computer 150 is connected to the speaker 159 by a speaker line 
out cable 161. 

25 [021] In an alternative aspect of the present invention, the apparatus also 

has an external MIDI sound card 155 and a MIDI sound module 170. According 
to this embodiment, the processing computer 150 is connected to the external 
MIDI sound card 155 by a USB cable 156. The MIDI sound card 155 is 
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connected to the MIDI sound module 170 via a MIDI cable 42. The MIDI sound 
module 170 is connected to the internal sound card 148 via an audio cable 158. 
[022] In a further alternative embodiment, the apparatus has a lighting 

controller 160 controlling a set of lights 162. The lighting controller 160 is 
5 connected to the processing computer 150. The lighting controller 160 is also 

connected to each light of the set of lights 162. The lighting controller 160 can be 
any known apparatus for controlling a light or lighting systems. The set of lights 
162 can be one light. Alternatively, the set of lights 162 can be comprised of any 
number of lights. 

10 [023] In one embodiment, the actuator 30 may be any known mechanical 

contact switch that is easy for a user with disabilities to operate. Alternatively, 
different types of actuators, for example, light sensors, may also be used. In one 
aspect of the present invention, the number of actuators 30 can vary according to 
factors such as the user's skill level and physical capabilities. While FIG. 1 

15 shows an embodiment having a single actuator 30 on the user console 20, further 

embodiments may have a plurality of actuators 30. 

[024] According to one embodiment, the processing computer 150 may 

be any standard computer, including a personal computer running a standard 
Windows® based operating system, with standard attachments and components 
20 (e.g., a CPU, hard drive, disk and CD-ROM drives, a keyboard and a mouse). 

The processor 154 may be any standard processor such as a Pentium® processor 
or equivalent. 

[025] FIG. 1A depicts a schematic diagram of a music apparatus 11, 

according to an alternative embodiment of the present invention. The apparatus 
25 11 has a user console 20 with eight actuators 30 and a wireless transmitter 19, a 

converter 100 with a wireless receiver 17, and a processing computer 150. The 
actuators 30 are connected to the wireless transmitter 19 with actuator cables 31. 
In place of the electrical connection between the actuator 30 and the voltage 
converter 100 according to the embodiment depicted in FIG. 1, the wireless 
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transmitter 19 shown in FIG. 1 A can transmit wireless signals, which the wireless 
receiver 17 can receive. 

[026] FIG. 2 is a flow diagram showing the operation of the apparatus 

10, according to one embodiment of the present invention. The user initiates 
5 operation by pressing the actuator button 3 1 (block 60). Upon engagement by the 

user, the actuator 30 transmits an actuator output signal to a voltage converter 100 
through the actuator cable 35 (block 62). Alternatively, the actuator 30 transmits 
the output signal to the wireless transmitter 19, which transmits the wireless 
signal to the wireless receiver 17 at the voltage converter. The voltage converter 

10 100 receives the actuator output signal 36 and converts the actuator output signal 

36 to a voltage converter output signal 146 (block 64). The voltage converter 
output signal 146 is in the form of a serial data stream which is transmitted to the 
processing computer 150 through a serial cable 145 (block 66). At the processing 
computer 150, the serial data stream is processed by the software 152, converted 

15 into an output signal, and transmitted to the speaker 159 to create sound (block 

68). In accordance with one aspect of the invention, the serial data contains 
further information that is further processed and additional appropriate action is 
performed (block 70). That is, the additional action message information 
contained in the data stream is read by the software 152, which then activates the 

20 appropriate hardware to perform the additional required action. 

[027] According to one embodiment, the step of processing the serial 

data stream, converting it into an output signal, and transmitting the signal to a 
speaker 159 to create sound (block 68) involves the use of a known 
communication standard called a musical instrument digital interface ("MIDI"). 

25 According to one embodiment, the software 152 contains a library of preset MIDI 

commands and maps serial data received from the voltage converter output signal 
146 to one or more of the preset commands. As is understood in the art, each 
MIDI command is sent to the MIDI driver (not shown) of the processing 
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computer 150. The MIDI driver directs the sound to the internal sound card 148 
for output to the speaker 159. 

[028] Alternatively, the MIDI command is transmitted by the MIDI 

sound card from the processing computer 150 to the MIDI sound module 170. 
5 The MIDI sound module may be any commercially-available MIDI sound module 

containing a library of audio tones. The MIDI sound module 170 generates a 
MIDI sound output signal which is transmitted to the processing computer 150. A 
signal is then transmitted to the speaker 159 to create the predetermined sound. 
[029] FIG. 2A is a flow chart depicting the activation of the additional 

10 action of launching a web browser, according to one embodiment. The software 

152 processes the further information in the serial data stream relating to 
launching a web browser (block 72). A signal is then transmitted to the browser 
software 152 indicating that the browser should be launched (block 74). The 
browser is launched and displayed on the monitor 180 (block 76). According to 

15 one embodiment, the browser then displays images as required by the data stream 

(block 78). For example, photographs or pictures relating a story may be 
displayed. Alternatively, the browser displays sheet music coinciding with the 
music being played by the speaker 159 (block 80). In a further alternative, the 
browser displays text (block 82). The browser may display any known graphics, 

20 text, or other browser-related images that may relate to the notes being played by 

the speaker 159. In an alternative aspect of the present invention, the browser is 
an embedded control within the software 152 of the processing computer 150. 
[030] FIG. 2B is a flow chart depicting the activation of the additional 

action of displaying a graphical keyboard, according to one embodiment. The 

25 software 152 processes the further information in the serial data stream relating to 

displaying a graphical keyboard (block 84). A signal is then transmitted to the 
appropriate software 152 indicating that the keyboard should be displayed (block 
86). The keyboard is displayed on the monitor 180 (block 88). According to one 
embodiment, interaction is then provided between the sounds emitted by the 
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speaker 159 and the keyboard (block 90). According to one embodiment, the 
interaction involves the highlighting or otherwise indicating the appropriate key 
on the keyboard for the note currently being emitted by the speaker 159. 
Alternatively, any known interaction between the sound and the keyboard is 
5 displayed. 

[031] FIG. 2C is a flow chart depicting the activation of the additional 

required action of displaying a music staff, according to one embodiment. The 
software 152 processes the further information in the serial data stream relating to 
displaying a music staff (block 92). A signal is then transmitted to the appropriate 

10 software 152 indicating that the music staff should be displayed (block 94). The 

music staff is displayed on the monitor 180 (block 96). According to one 
embodiment, interaction is then provided between the sounds emitted by the 
speaker 159 and the music staff (block 98). According to one embodiment, the 
interaction involves the displaying the appropriate note in the appropriate place on 

15 the music staff corresponding to the note currently being emitted by the speaker 

159. Alternatively, any known interaction between the sound and the music staff 
is displayed. 

[032] FIG. 2D is a flow chart depicting the activation of the additional 

action of displaying lights, according to one embodiment. The software 152 

20 processes the further information in the serial data stream relating to displaying 

lights (block 200). A signal is then transmitted to the lighting controller 160 
indicating that certain lights should be displayed (block 202). Light is displayed 
at the set of lights 162 (block 204). According to one embodiment, interaction is 
then provided between the sounds emitted by the speaker 159 and the lights 

25 (block 206). According to one embodiment, the interaction involves the flashing 

a light for each note emitted by the speaker 159. Alternatively, any known 
interaction between the sound and the lights is displayed. 

[033] FIG. 3 depicts the structure of a voltage converter 100, according 

to one embodiment of the present invention. The voltage converter 100 has a 
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conversion section 102, a microcontroller section 120, a RS232 output 140, and a 
power supply 101. In operation, the conversion section 102 receives the actuator 
output signal 36 from a user console 20. According to one embodiment, the 
conversion section 102 recognizes voltage change from the actuator 30. The 
5 microcontroller section 120 polls for any change in voltage in the conversion 

section 102. Upon a recognized voltage change, the microcontroller section 120 
sends an output signal to the RS232 output 140. According to one embodiment, 
the output signal is a byte representing an actuator identifier and state of the 
actuator. According to one embodiment, the state of the actuator information 
10 includes whether the actuator is on or off. The RS232 output 140 transmits the 

output signal to the processing computer 150 via 146. 

[034] FIG. 4 depicts a perspective view of another embodiment of the 

present invention. Referring to FIG. 4, the present invention in one embodiment 
includes a user console 20, mounted on an adjustable support 50. In this 
15 embodiment, the user may adjust the height of the user interface table by raising 

or lowering the support. Alternatively, the music apparatus may utilize any other 
known support configuration. 

[035] FIG. 5 shows a cross-section of a user console 20 according to one 

embodiment of the present invention. The console 20 has a console bottom 

20 portion 21 sized to store a plurality of actuators. In one embodiment, a console 

top portion 22 with cutout 28 is attached to the user console bottom portion 21. 
Cutout 28 provides access to the interior 24 of the user console 20 through an 
opening 29 in the user console top portion 22. At least one actuator 30 is attached 
to the user console top surface 34 by an attachment means 23 that holds the 

25 actuator 30 in place while the apparatus is played but allows the musician to 

remove or relocate the actuator 30 to different positions along the user console top 
surface 34 and thus accommodate musicians with varying physical and cognitive 
capabilities. In one embodiment, attachment means 23 may be a commercially- 
available hook-and-loop fastening system, for example Velcro®. In other 
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embodiments, other attachment means 23 may be used, for example, magnetic 
strips. An actuator cable 35 is routed into the interior 24 of the user console 20 
through the opening 29. Alternatively, a plurality of actuators 30 can be used, and 
unused actuators can be stored in the user console interior 24 to avoid cluttering 
the user console top surface 34. 

[036] According to one embodiment in which the user console top 

portion 22 is rigidly attached to the user interface table bottom portion 21, the 
user console 20 is attached to an upper support member 51 at the table support 
connection 26 located on the bottom surface 27 of the user console top portion 22. 
[037] Although the present invention has been described with reference 

to preferred embodiments, persons skilled in the art will recognize that changes 
may be made in form and detail without departing from the spirit and scope of the 
invention. 



-10- 



